The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus capable of outputting an image having a wide dynamic range by using a solid state image device to synthesize and compress images of two or more screens having different quantities of exposure.
Hitherto, the upper limit for an image pickup dynamic range (a luminance range in which an image can be pick up) of an image pickup device, such as a television camera, is determined in accordance with the saturation level of the image pickup device. On the other hand, the lower limit of the same is determined in accordance with the noise level of the image pickup device and its peripheral circuits. The dynamic range of a usual image pickup device is about 50 dB to about 60 dB, while that of a TV (television) monitor or the like is about 45 dB. On the other hand, the dynamic range of a usual object is wider than the above-mentioned values and thus the same reaches 80 dB to 100 dB. Therefore, if a usual object is picked up by the image pickup device and the image of the object is displayed on a monitor or the like, a bright portion and a dark portion of an image having a contrast cannot simultaneously be displayed. Since the image pickup dynamic range is usually smaller than the luminance range for an object, a bright portion is whitened excessively and a low bright portion is blackened unsatisfactorily.
Accordingly, if a usual object is picked up by the image pickup device and the image of the object is displayed on a monitor or the like, a bright portion and a dark portion of an image having a contrast cannot simultaneously be displayed. In order to solve this problem, techniques have been disclosed in, for example, Jpn. Pat. Appln. KOKAI Publication No. 5-167889 and Jpn. Pat. Appln. KOKAI Publication No. 6-319149.
The apparatus disclosed in Jpn. Pat. Appln. KOKAI Publication No. 6-319149 has a structure such that the image of an object allowed to pass through an image pickup optical system is divided into two directions by a half mirror or the like. Then, a filter or the like is used to obtain, from the divided image signals, an image signal in which a dark portion of the object is deformed and a bright portion is not saturated and thus picked up satisfactorily is output and another image signal in which a bright portion is saturated and a dark portion is not deformed and thus picked up satisfactorily is obtained. The two image signals are synthesized so that an image signal having information from the dark portion to the bright portion can be obtained. The image signal is logarithmically compressed, and then subjected to a filtering process so that the dynamic range and the gain of the signal are controlled. Then, the image is output.
The apparatus disclosed in Jpn. Pat. Appln. KOKAI Publication No. 5-167889 has a structure such that an average value and a standard deviation of image data are used to calculate coefficients and add and multiply the coefficients so as to control the dynamic range and the gain. As a result of the foregoing method, an image having a wide latitude and excellent contrast can be displayed on the monitor.
Moreover, the following technique has been disclosed in Jpn. Pat. Appln. KOKAI Publication No. 57-39673: the shutter speed is controlled or an ND filter is used to change the quantity of exposure of the image pickup device, and then two obtained images having different quantities of exposure are synthesized so that the image pickup dynamic range is widened.
In Jpn. Pat. Appln. KOKAI Publication No. 7-75026, a technique has been disclosed in which the characteristic at a connection point of wide dynamic range signals obtained by synthesizing two or more images having different quantities of light are made to be adequate.
A technique is known which provides a knee characteristic as shown in FIG. 1 in order to compress a wide dynamic range signal.
However, the techniques disclosed in Jpn. Pat. Appln. KOKAI Publication No. 57-39673 and Jpn. Pat. Appln. KOKAI Publication No. 7-75026 have no contrivance to solve a problem in that the noise level at the connection point is rapidly changed. When two images having different quantities of exposure are synthesized, noise is simultaneously amplified because an image signal having a smaller quantity of exposure is multiplied with an exposure quantity ratio with the image signal having a larger quantity of exposure as shown in FIG. 2.
In an example case where images photographed with quantities of exposure respectively realized by shutter speeds of 1/60 second and 1/2000 second are synthesized, the signal obtained at the shutter speed of 1/2000 second is amplified to 32 times. At this times, also the noise level is amplified to 32 times. Even if the dynamic range of the image to be picked up is not considerably wide, the noise level is amplified to 32 times. Therefore, the noise level is rapidly raised at the connection point and thus the quality of the image deteriorates excessively. The above-mentioned problems become more critical as the exposure quantity ratio is raised in order to widen the image pickup dynamic range.
The method for synthesizing two images having different quantities of exposure shown in FIG. 2 and arranged to provide the knee characteristic for the synthesized wide dynamic range signal is permitted to add two image signals having different quantities of exposure to realize the knee characteristic. It is preferably that two image signals having different quantities of exposure be switched because the S/N ratio deteriorates by about 3 dB. A case where the quantity of exposure is made such that the dynamic ranges are fixed to 1/60 second and 1/2000 second is considered. Since the dynamic range is fixed to 1/2000 second even if the dynamic range of the object is not considerably wide and the adequate value of the high shutter speed is, for example, 1/200 second, an image having a bright portion having an excessively low contrast is unintentionally formed.
The techniques disclosed in Jpn. Pat. Appln. KOKAI Publication No. 6-319149 and Jpn. Pat. Appln. KOKAI Publication No. 5-167889 have no contrivance to output a moving image in accordance with a calculation time period of the coefficients for controlling the dynamic range and the gain by calculating the average value and the standard deviation from image data. In a case where the brightness of the image is always changed as a moving image, a long time is required to perform the calculation of an image (for example, in a case where a light source, such as stroboscope is operated) having the brightness which is rapidly changed. Thus, there arises a problem in that an excessively long time is required to display an image having a wide latitude and excellent contrast.